Adaptive ' random data generation for computer software testing

This paper discusses computer-assisted generation of input data for program testing. The test data generation system must produce input data that is per specification, unbiased by prior analysis of the program to be tested. The test data generators evaluated use random generation techniques to produce the test data. To reduce the number of sets of test data needed to test a program, summary information aboutthe performance of previously generated sets of input data is used to modify the probability distributions upon which the next set of test data is based. Four test data generators were evaluated and used to generate test data to exercise five testcase programs of various complexity. Observations of the results of the actual evaluation runs and of the types of structures involved led to the establishment of some guidelines for future testing operations. Program verification of any sort was not attempted. Manual checks of the normal program outputs did detect a number of software errors. No attempt was made to automatically isolate or even detect software faults. Nor was any comparison with other methods, manual or automatic, made. The Software Testing System developed to use as a tool to evaluate the test data generators is similar to others previously reported. 17 These systems require repetition of steps to guide and assist test case selection, to execute the instrumented program and to analyze testing coverage until the test goals have been achieved. The system used here to evaluate the test data generators not only provided the summary reports of testing progress similar to the above systems, but also provided progress summary information to the test data generators which was then used to modify the probability distributions controlling the generation of the test data. Most previous work relating to the generation of test data for program testing in some way base the test data on analysis of the program to be tested. R. L. Sauders determined the format and description of the test data by analysis. Other systems912 use analysis of the program to guide choice of test data when the main goal of testing was usually to execute all paths through a program.